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Product  Inequalities  Involving  the 


Multivariate  Normal  Distribution 


Abstract 

Suppose  Y'  = (Yj,...,Y£)  possesses  a multivariate  normal 

distribution  with  mean  vector  0 and  positive  semidefinite 

pi 

covariance  matrix  Z . If  <=  R denote  convex  regions 
symmetric  about  the  origin,  then  conditions  are  given  such  that 


P(Y  e C.  , i = 1 k)  > n P (Y . e C.  ) 

11  i=l  1 1 


and/or 


P(Y.  eCif  i = l,...,k)  * _n  P(Yi  e^) 


obtain.  These  conditions  imply  that  chi-squared  random  variables 
defined  from  a multivariate  normal  distribution  are  always 
positively  dependent  and  non  negatively  correlated.  Other  appli- 
cations involve  conservative  simultaneous  confidence  regions  in 
a multivariate  regression  setting. 


AMS  Classification  numbers:  Primary  62H99;  Secondary  62E10. 

Key  words  and  phrases:  product  inequality,  multivariate  normal, 
Wishart  matrix,  simultaneous  confidence  intervals 
in  multivariate  regression. 


INTRODUCTION. 


I 


1. 

The  following  question  has  received  the  attention  of 

several  investigators.  Suppose  Y'  = (Y*,  Y^, , YR') 

possesses  a multivariate  normal  distribution  with  mean  vector 

0 and  positive  semidefinite  (p.s.d.)  covariance  matrix  1,  j.e.  Y 

is  n(0,>:)  . The  dimension  of  Y.  is  p.  so  that  the  dimen- 

k 1 

sion  of  Y must  be  1 p.  . We  let  C.  denote  a convex  re- 

1 1 1 

gion  in  (and  CM  its  complement)  which  is  symmetric  about 

the  origin  (x  e Ci  implies  - x c O)  . Under  what  conditions 
will  at  least  one  of  the  two  inequalities 


(1.1) 

and 


k 

P1Y.  e C.,  i « 1 k)  > 11  P ( Y • e C-) 

i = 1 1 1 


(1.2) 


P ( Y . c r.  , i 
v~  1 1 ’ 


k 

1 k)  2 n P (Y . . C.  ) 

i = 1 1 1 


hold?  A primary  application  of  such  inequalities  has  been 
obtaining  conservative  simultaneous  confidence  bounds  which 
are  easily  computable  although  the  inequality  is  useful  in 
other  areas  as  well. 

Dunn  (1958)  showed  the  inequality  (1.1)  would  hold  if 
p^  = 1 for  all  i and  the  C.  were  equal,  providing 
k * 2 or  k = 3 . (For  k = 2 it  is  easily  shown  that 

(1.1)  and  (1.2)  are  equivalent.)  Moreover  she  proved  her 
conjecture  for  arbitrary  k if  the  correlation  matrix 

corresponding  to  J!  was  of  the  form  p..  = b.b. 

ij  i J 

i ¥ i and  0 < b.  < 1 . i = 1 , . . . ,k  . 


, i , .i  - 1 , . . . , k , 
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S i dak  ( 1 ‘.)  b 7 ) was  able  to  extend  Dunn's  results  to  the 
case  of  an  arbitrary  covariance  matrix  and  different  C^'s 
(although  still  one  dimensional). 

Scott  (19b7)  purported  to  prove  tiie  same  result  as  Sidak 
as  well  as  inequality  (1.2)  with  the  same  conditions  holding. 
However  a subtle  conditioning  error  occurs  in  Scott's  argu- 
ments and  invalidates  his  proofs.  Moreover,  Sidak  (1971)  has 
constructed  a counterexample  to  Scott's  second  inequality  so 
that  (1.2)  does  not  hold  under  these  conditions. 

In  light  of  Sidak's  1967  results  it  is  tempting  to  con- 
jecture an  analogue  of  a one  sided  result  due  to  Slepian  (1962), 
namely  that  decreasing  the  absolute  value  of  the  correlations 

should  decrease  P(c.  < Y.  sc.,  i = 1,  . . . , k)  . However 

i i i 

a counter  example  of  Sidak  (1968)  showed  that  this  is  not 
true  in  general.  He  did  show  however  that  the  conjecture 
held  if  the  absolute  values  of  the  correlations  decreased 
in  a particular  manner. 

Khatri  (1967)  was  able  to  prove  (1.1)  for  higher  dimen- 
sions (although  the  p.  must  be  equal)  when  the  C.j's  are 
ellipsoids  of  the  form  {x*x'Axsc^}  , the  covariance  structure 
of  Y is  the  Kronecker  product  k jPM1  f and  ^ and 

are  positive  semi de finite . 

By  adding  a restriction  similar  to  Dunn's  on  the  matrix 
Y , Khatri  was  able  to  prove  (1.2)  in  the  same  setting. 

He  also  considered  some  situations  where  the  ellipsoids  are 
random.  Later  Khatri  (1970)  had  apparently  extended  his 
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1967  results  to  the  point  that  (1.1)  and  (1.2)  hold  without 
any  restrictions  on  I or  the  ' s (other  than  those  ini- 
tially assumed).  However  Sidak  (1975)  isolated  an  unobtrusive 
error  in  both  Scott's  1967  paper  and  Khatri's  1970  paper.  More- 
over, Sidak's  1971  counterexample  lias  shown  that  the  general 
inequality  (1.2)  without  restrictions  is  incorrect.  Has 
Gupta  et.  al.  (1972)  considered  inequalities  of  the  type 
(1.1)  and  (1.2)  in  the  more  general  setting  of  elliptically 
contoured  distributions.  They  were  able  to  establish  in- 
equality (1.1)  with  out  restrictions  on  l providing  every 
P^  exceptone  equals  one. 

Tong  (1970)  and  Sidak  (1973)  have  established  some 
inequalities  related  to  (1.1)  and  (1.2)  when  all  the  corre- 
lations determined  by  E are  identical. 

Whether  inequality  (1.1)  holds  in  general  without  addi- 
tional assumptions  has  been  an  open  question  discussed  in 
several  of  the  above  references.  It  is  the  purpose  of  this 
paper  to  give  some  new  conditions  under  which  inequalities 
(1.1)  or  (1.2)  hold  and  discuss  briefly  a few  applications 
which  result. 

2.  THEOREMS  AND  PROOFS. 

We  will  have  need  of  the  following  lemmas  in  proving 
our  theorems. 

LEMMA  1.  If  l'  « (Zlf  ...,  Z ) is  a random  vector  with 
density  f(z)  symmetric  about  0 ( f ( z ) = f(-z)  for  all 
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z)  such  that  {z;  f(z)  < c}  is  convex  for  ail  non-negative 
c , a is  an  arbitrary  fixed  vector  in  Ir  , and  C is  a 
convex  set  symmetric  about  the  origin,  then  P(Z  i C + \a) 
is  a non increas ing  function  of  A ( 0 s A < »)  . 

This  lemma  is  due  to  T.  A.  Anderson  f 19  5 5)  and  is  well 
known . 


1.1:  MM  A 2. 


partitioned  as 


Suppose 


the 

M 

A,  J 


squa re , 
whe  re 


s y m me  trie  ma t r i x A is 
Ajj  and  A,,  (nons insular 


are  square.  Then  A is  positive  definite  (p.d.)  iff  both 
'll  " ;1IU*  ' ’ ’ are  positive  definite. 


PROOF . It  is  well  Known  that  if  ii  is  a p.d.  matrix 
and  D is  nonsingular  (same  dimension),  then  DHD'  is  p.d. 
1 f we  de  f i ne  t h e ma  t r i x 


B = 


1 2 A 2 2 


l 


t 


then  B is  clearly  nonsingular.  However  then 


BAB  ' 


0 


0 


» 


which  implies  the  desired  result. 


LEMMA  3.  If  h^(x),  h,(x)  are  nonnegative  functions 
either  both  non increas ing  or  both  nondecreasing  and  \ is 
a random  variable,  then 


-s- 


E hj(X)  h2(X)  s E hx  (X) -15  h2  (X)  . 

This  result  is  well  known  and  appears  in  various  places, 
one  place  being  Kimball  (1951). 

LEMMA  4.  If  x = (Xj,  ...»  x ) is  a p-variate  vector 

and  4>(x)  is  a real-valued  function  invariant  under  orthogonal 

transformation  (4>(Ax)  = 4>  (x)  for  every  orthogonal  matrix 

2 

A),  then  <Hx)  is  a function  of  x only  through  Ex^  . 


PROOF.  For  a given  vector  x = (Xj,  ...»  x )'  , let 
0(x)  denote  an  orthogonal  matrix  such  that  0(xl  x = 

((Exp*5,  0 0)  " . (WLOG  we  may  assume  0(ax)  = 0(x) 

for  any  positive  constant  a.)  The  assumption  then  guarantees 
that  4>  (x)  = 4>(0(x)x)  is  a function  of  only  Ex?  . 

Theorem  1 is,  in  one  sense,  a generalization  o<"  a theorem 
by  Das  Gupta  et.  al.  (1973). 

1 1 

THEOREM  1.  Suppose  (v,  ) is  n(0,  E)  with  covariance 

i?  T.  T. 

- Ill7 

matrix  E expressable  as  E = (T  T ”)  and  that  C is  an 

1 2 1 1 2 2 

arbitrary  convex  set  symmetric  about  0 . Then  if  A is 
ideinpotent  (A2  = A)  , 

(2.1)  P(Y.eC,  Y;AY2  < c)  i PfYjCC)  P(Y,'AY,  s c) 


f o rail  c l)  . 


P RO 01.  by  making  linear  transformations  on  Yj  and 
'i  , which  may  reduce  the  number  of  random  variables,  we  may 
assume  WI.OG  that  both  .V  ^ ^ and  I,,  are  of  full  rank  and 
that  the  left  side  of  (2.1)  is 

P(Yj  , Y'  Y , c)  . 

he  nav  also  assume  ihH',  that  is  full  rank,  since  if  it 

is  not,  we  may  define  a sequence  of  positive  definite  matrices 
of  the  same  form  which  converge  elementwise  to  and  then 

use  a limiting  argument  to  obtain  the  desire  1 conclusion. 

3v  a continuity  argument,  we  may  replace  I,-,  by 
l l - til,,  and  still  have  I be  positive  definite  if  t; 
is  sufficiently  small.  Thus  the  augmented  matrix. 


(2.2) 


■ * = I r 


l,  2 (1-e) I 

(I-e)I,-,  (l-e)I,,  / 


is  positive  definite  for  c sufficiently  small. 

To  see  this,  partition  off  the  last  row  and  column  ot 


submatrices  and  note  from  Lemma  2 that 


is  p . d . iff 


I 

I 

and  (l-e)I7?  are  p.d. 

Suppose  now  that  (Y',  Y7,  Z')  ' is  n(0,  Z*).  Then 
we  may  write 

PdjtC,  Y7Y7  - c)  = E P ( Y t e C , Y7'Y2  • c|Z) 

(2.4)  = ElP(Y1c C| Yj  is  n ( Z 1 2 Z/ ( 1 - e) , x ^ ( 1 - e ) ) • 

I’CY^Y,  • c|Y2  is  n(Z,  el22)  } 

where  Z is  n (0 , ( 1 - t ) I , -,)  . If  we  let  |Z|  = (EZ?)1* 
denote  the  distance  from  Z to  the  origin,  we  may  express 
Z = Q C | Z ! , 0,...,  0)  where  the  random  orthogonal  matrix  Q 
and  | Z | are  independent.  Moreover,  since  the  distribution 
of  Q is  invariant  if  multiplied  on  the  left  by  a fixed 
orthogonal  matrix,  Q must  posess  the  ’.laar  invariate  distri- 
bution discussed  in  Anderson  (1958).  Since  the  conditional 
distribution  of  Y,'Y7  is  (except  for  a constant)  a noncen- 
tral chi  squared  distribution  with  noncentrality  parameter 
I Z | * , and  hence  free  of  Q , we  may  express  the  quantity  in 

(2.4)  as 

(2.5)  E IE  P(Y  eC|Q,|Z|)]-P(Y'Y?  sc||Z|)]  . 

-I  r\  i *.4.  ~ 


I1 


3y  Lemma  1,  the  second  factor  of  the  integrand  is  a decreas- 
ing function  of  |Zj  . Similarly,  for  a fixed  value  of  Q , 


• /Om 


r 


the  integrand  of  E is  nonincreasing  in  |Z|  , and  hence 

Q 

P(Y^eC|Q,  | Z | ) is  nonincreasing  in  |Z|  . Then  by  apply- 
ing Lemma  3,  the  expression  in  (2.5)  must  be 


> E P(Y  ,cC|Q,  | Z |)  • H P(Y;  Y7  s c | | Z|  ) 
1 2 [ ,Q  \Z\  “ ~Z 


= P(YlEC)-P(Y2"Y2  S c) 


which  was  the  desired  result. 

3y  replacing  the  words  f'nonmcreasing  in  j Z | " by 
"nondecreas ing  in  |Z|"  the  following  corollary  is  immediate 


COROLLARY  1.  Under  the  same  assumptions  as  in  Theorem  1, 


P (Y  x l C , Y'AY2  > c)  > P(Y1eC)P(Y2'AY2  2 c)  . 

I f we  assume  that  C is  of  the  form 

C = i)L,  s ci> 

where  the  A^  are  idempotent  matrices,  then  repeated  appli- 
cation of  Theorem  1 gives  the  following  corollary. 

COROLLARY  2.  If  Y " = (Y',  . . . , Y£)  is  n(0,£)  where 
the  corresponding  partition  of  Z is 


-9  - 


then 


£ > 


/ 


1 1 


\ 


(2.6) 


P(Y.*A.  Y. 

~ l i ' i 


°i  • 1 


k) 


i-  1 


PQ'.'AV.  c.) 

v 4 i i 4 i i 


for  all  positive  constants  Cj,  c.  . 

I'll  is  is  a generalization  of  Si  dak's  ( 1 9 0 7 ) inequality 
in  the  sense  that  a symmetric  interval  generalizes  to  a 
symmetric  spheroid  in  higher  dimensions  and  a n(0,l)  random 
variable  generalizes  to  a n(0,I)  random  vector.  Corollary 
2 states  that  the  chi  squared  random  variables  V-  AY.  are 
positively  orthant  dependent.  Stronger  forms  of  dependency 
such  as  associativity,  positive  regression  dependence  and 
monotone  likelihood  ratio  dependence  as  discussed  for  example 
in  Dykstra,  Mewet t , and  Thompson  (1973)  do  not  hold  in  general 
for  these  chi  squared  variables. 


However,  when  k * 2,  more  can  be  said  about  the  nature 
of  the  inequality  in  Corollary  2 as  indicated  in  the  following 
t heorem . 

THEOREM  2.  If  (d')  i s n(0,S),  where  may  be 

I -) 

partitioned  as 


tin'll  for  all  c.  and  c, 


(2.') 


P(Y{Y1  - c,,  v;v2  • c,) 


is  a nondoc rens i n g function  of  the  characteristic  roots  of 


I’ROOI- . There  exist  orthogonal  matrices  Q.  ami  Q,  such 


lV  12^2  * llia8^'  ] » • ■ • • 


where  p i * • • • * ;n'°  the  characteristic  roots  of  y | ^ . Since 

we  may  express  (.2.71  as 


l 2 . S 1 


1 , 2 , 

I i' ( v | . • C - s Y“  , yf,  • C,  - y‘  V“2  | Y*.Y*) 
Y * Y * 11  1 iai  11  1 - - 1 *•  1 *• 


where  lY*,l*)  denote  the  random  vector.}  (Y,,Y,1  with  the  first 
components  removed,  if  we  increase  only  Oj  . it  is  well  known  that 
the  integrand,  and  hence  the  whole  expression  must  increase. 

Since  p.  is  not  special,  the  theorem  easily  follows. 

In  attempting  to  extend  Corollary  2 to  ellipsoidal  regions 
rather  than  spheroids,  difliculties  are  encountered.  However,  by 
putting  a rather  stringent  condition  on  the  covariance  structure, 
a result  rather  similar  to  Miatri’s  (IPo")  result,  though  stated 
differently,  is  possible 

TllliORIlM  3.  l et  Y ’■  (Yj’,.  . . , Y^l  denote  <t  kp  x l random 

vector  possessing  an  n(0,  lt«  ;:)  distribution  where 


(2.9) 


B » Z = 


bllZ  b12E 


b22E 


blkZ 


bkkL 


is  the  Kronecker  product  of  B and  Z . Then  if  A is  a 
symmetric  p.d.  matrix, 

-1  k i 

(2.10)  P (Y T A Y.  sc.,  i = l k)  >n  P(Y."  A Y.  sc.) 

ill  ^ ~i  ~ l l 

for  all  positive  constants  c^,  ...,  . 

PROOF.  We  may  assume  WLOG  that  B and  I are  full 
rank.  Then,  by  using  the  nonsingular  matrix  Q such  that 

Q I Q'  = I and  Q A Q'  = diag  A = (A^ 


where  A^  are  the  roots  of  |A 
(2.10)  as 


(2.11)  PQTA'V  s cL,  i = 1, 


A E | = 0 , we  may  express 


,k)  > n P ( Y A 1 Y - s C.) 
1 l 1 i 


where  Y has  covariance  matrix  B 0 1 . The  augmented 


matrix 


bn1 


b22  1 


blk 1 blk! 


* bkkIf1_c)bkk1 

(l-e)b..  I 


will  be  p.d.  if  e >0  is  sufficiently  close  to  0.  Thus  if 


(Yj,  ....  Yk,  Z ')  ' is  n(0,  11*  s I),  tlic  left  side  of  (.2.11 
is  exp  res sable  as 


\:  l’CX  *A‘ AYi*.ci,i“l k-lU).l'fYk'A'1Yk*.ck|Z: 


However,  by  the  diagonal  structure  of  A 1 and  the  condi- 
tional covariance  structure  of  Y given  Z , it  follows 
from  Lemma  1 that  each  factor  of  the  integrand  is  a non- 
increasing function  of  Zj  when  Z^  , . ..,  Z}  are  held 
fixed.  Repeated  applications  of  Lemma  3 imply  the  desired 
result . 

One  might  hope  that  Theorem  3 would  extend  in  certain 
situations  to  the  case  when  A is  random.  Under  the  right 
conditons,  this  does  indeed  happen;  the  right  conditions 
being  that  A possess  a central  Wishart  distribution  with 
covariance  matrix  Y and  that  A be  independent  of  Y . 


COROLLARY  3.  If  Y ' = (Y  * , . . . , Yj* ) is  n (0 , B »>:)  , ; 
has  a central  Wishart  distribution  with  covariance  matrix  )'. 
(S  is  W(v,E))  , and  Y and  S are  independent,  then 

- 1 k - 1 

(2.12)  P(Y.S  1Yisci,  i-l k)2llP(Y.S  ‘Y.sCj)  for 

all  positive  constants  c^ , c9,...,  ck  . 


PROOF.  Clearly  we  may  assume  £ = I 1VLOG.  Then  the 
conditional  distribution  of  (Y.'S  W.,...,  Y.'S  Y.)  given 


I 

: 
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S depends  upon  S only  through  its  characteristic  roots 

^ “ ( 'I'j . • • • . •J'p)  • 

3y  Theorem  3 

p C Y r S*  1 Y i S c.,  i « 1 k) 

* E P(Y.  S" 1 Y . s c.,  i = 1 k|*) 

♦ 

k . , 

(2.13)  * E n P(Y.  S 1 Yi  * c • | *)  • 

$ 1 1 11 

Since  the  charactertic  roots  of  a Wishart  matrix  with 
identity  covariance  matrix  are  stochastically  increasing  in 
sequence  (Dykstra  and  Mcwctt  (1978)),  and  since  each  factor 
of  the  integrand  in  (2.14)  is  nondecreasing  in  ij*.  , Theorem  1 
of  Dykstra,  ’lewett  and  Thompson  ( 1973)  preserves  the  desired 
inequality  when  the  product  sign  is  brought  outside  the  es- 
pectation  sign. 

3.  APPLICATIONS. 

Since  the  right  side  of  the  (2.0)  is  just  the  product 
of  central  chi-squared  probabilities,  Corollary  2 essentially 
states  that  chi-squared  random  variables  which  are  quadratic 
forms  of  a multivariate  normal  vector  are  always  positively 
orthant  dependent  as  defined  in  Dykstra  et.  al.  (1973). 

However  an  example  of  Sidak’s  (1971)  shows  that  stronger 
forms  of  positive  dependence  like  "association",  stochasti- 
cally increasing  in  sequence"  and  "positively  likelihood  ratio 
dependence"  do  not  bold  in  general.  However,  positive  orthant 
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dependcnce  does  imply  non-negat i vc  correlations  by  the  ex- 
pression lor  the  covariance  given  in  l.chmann  (1966). 

(.)  Corollary  2 seems  somewhat  related  to  the  bivariate  chi- 
squared  inequality  given  by  Jensen  (19691.  'lowever  Jensen's 
inequality,  while  two-sided,  only  hold  tor  k * 2 , equal 
degress  of  freedom,  and  identical  intervals.  Moreover,  since 
bivariate  chi -squared  random  variables  defined  in  this  manner 
are  conditionally  independent  and  identically  distributed  as 
shown  In1  Shaked  (19771 , Jensen's  inequality  would  also  hold 
for  any  Borcl  set. 

(It)  Corollary  2 implies  that  the  product  of  the  marginal 
c.d.f.'s  of  multivariate  chi-squared  random  variables  serves 
as  a lower  bound  for  the  joint  c.d.f.  of  the  random  variables. 
A similar  statement  applies  for  multivariate  F random 
variables  such  as  discussed  by  Schuurinann,  Krishnaiah  and 
Chat topadhyay  (1975).  This  follow  by  conditioning  on  the 
independent  denominator,  applying  Corollary  2,  and  then  using 
Lemma  3. 

(c)  If  goodness  of  fit  statistics  are  defined  on  different 
co-ordinates  of  multivariate  data,  then  the  asymptotic  chi- 
squared  distributions,  assuming  the  null  hypotheses  to  be 
true,  will  satisfy  the  inequality  in  Corollary  2.  Thus  if 
one  rejects  the  hypothesis  that  all  univariate  hypotheses 
are  true  whenever  a univariate  hypothesis  is  rejected,  he 
wi 1 1,  asympt ot ica 1 ly , have  a conservative  estimate  of  the 
significance  level  if  he  treats  the  individual  tests  as  being 


I 

I 

I 
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independent  . 

(d)  An  application  of  Corollary  3 involves  simultaneous  in- 
ference in  multivariate  linear  regression.  That  is,  let  Y 
be  a N * p data  matrix  of  N independent  observat ions  on 
p responses,  X be  a N * q design  matrix  of  fixed  known 
independent  variables,  3 be  a q * p matrix  of  parameters 
and  R be  a N * p matrix  of  random  errors  whose  rows  are 
distributed  independently  as  n(0,  E ) random  vectors.  It 
is  well  known  from  least  squares  theory  that  the  estimator 
of  B which  minimizes  Tr[(Y  - XB)  '(Y  - XR) ] is  given  by 

3 = (X'X)  ' 1 X *Y  . 

qxp 

assuming  X'X  is  of  full  rank.  The  distribution  of  the  row- 

A 

wise  rolled  out  version  of  3 is  then  multivariate  normal 
with  covariance  matrix  (X'X)"'»  E and  mean  equal  to  the 
row-wise  rolled  out  B . 

Moreover,  Qj.  = Y 'Y  - B'X'XB  is  independent  of  B and 
possesses  a central  Wishart  distribution  with  \ - q degrees  of 
freedom  and  covariance  matrix  E . Thus  if  we  construct 
ellipsoidal  confidence  regions  for  the  it^1  row  of  B based  i 

on  the  i row  of  B and  from  the  expression 

- Bl)  - B.)  S C.)  « 1 - . 

then  Corollary  3 guarantees  that  the  confidence  coefficient  for 
all  the  ellipsoidal  regions  to  contain  the  respective  para- 


This  generalizes  the 


meters  must  be  at  least  111  1 - a.)  . 
known  comparable  result  for  univariate  linear  regression, 
(o'  Siotani  (l‘dS9)  is  concerned  with 


where  Z^,  ....  Z^  is  a random  sample  from  a n(0,  E) 

distribution  and  S possesses  an  independent  K(v,E)  distribution 

A -> 

Siotani  approximates  the  distribution  of  Tjjj^  by  using 
Honfvrrom  inequalities.  However  the  product  inequality  given 
in  Fheorem  3 will  be  closer  to  the  true  probabilities  and  hence 
could  be  used  to  improve  Siotani's  approximations. 
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